• Two different strains of stickleback fish (ocean population (RS) and one from a freshwater lake (BP)) that you make microbe free.
• Microbes in the gut are known to interact with the gut epithelium in ways that lead to a proper maturation of the immune system.
• You carry out an experiment by treating multiple fish from each strain so that some of them have a conventional microbiota, and some are inoculated with only one bacterial species.
• You then measure the levels of gene expression in the stickleback gut using RNA-seq.
• You suspect that the sex of the fish might be important so you track it too.

Bernoulli distribution

• If the coin isn’t fair then \(p \neq 0.5\)
• However, the probabilities still sum to 1

\[ p + (1-p) = 1 \]

• Same is true for other binary possibilities
  • success or failure
  • yes or no answers
  • choosing an allele from a population based upon allele frequencies

Probability rules

• Now determine the probability if the H and T can occur in any order

\[ \text{Pr(H and T) =} \] \[ \text{Pr(X=H and Y=T) or Pr(X=T and Y=H)} = \] \[ (p*p) + (p*p) = 2p^{2} \]

• These are the ‘and’ and ‘or’ rules of probability
  • ‘and’ means multiply the probabilities
  • ‘or’ means sum the probabilities
  • most probability distributions can be built up from these simple rules

Joint and conditional probability

Joint probability

\[Pr(X,Y) = Pr(X) * Pr(Y)\]

• Note that this is true for two independent events
• However, for two non-independent events we also have to take into account their covariance

Joint and conditional probability

Conditional probability

• For two independent variables

\[Pr(Y|X) = Pr(Y)\text{ and }Pr(X|Y) = Pr(X)\]

• For two non-independent variables

\[Pr(Y|X) \neq Pr(Y)\text{ and }Pr(X|Y) \neq Pr(X)\]

• Variables that are non-independent have a shared variance, which is also known as the covariance
• Covariance standardized to a mean of zero and a unit standard deviation is correlation

Normal PDF | A function of two parameters

(\(\mu\) and \(\sigma\))

where \[\large \pi \approx 3.14159\]

\[\large \epsilon \approx 2.71828\]

To write that a variable (v) is distributed as a normal distribution with mean \(\mu\) and variance \(\sigma^2\), we write the following:

\[\large v \sim \mathcal{N} (\mu,\sigma^2)\]

Normal PDF

Why is the Normal special in biology?

Why else is the Normal special?

• The normal distribution is immensely useful because of the central limit theorem
  
• The mean of many random variables independently drawn from the same distribution is distributed approximately normally
• One can think of numerous situations, such as
  • when multiple genes contribute to a phenotype
  • or that many factors contribute to a biological process
• In addition, whenever there is variance introduced by stochastic factors or sampling error, the central limit theorem holds
• Thus, normal distributions occur throughout biology and biostatistics

z-scores of normal variables

• Often we want to make variables more comparable to one another
• For example, consider measuring the leg length of mice and of elephants
  • Which animal has longer legs in absolute terms?
  • Which has longer legs on average proportional to their body size?
  • Which has more variation proportional to their body size?
• A good way to answer this last question is to use ‘z-scores’

z-scores of normal variables

• z-scores are standardized to a mean of 0 and a standard deviation of 1
• We can modify any normal distribution to have a mean of 0 and a standard deviation of 1
• Another term for this is the standard normal distribution

\[\huge z_i = \frac{(x_i - \bar{x})}{s}\]

R Interlude | Complete Exercise 3.1

Components of hypothesis testing

• p-value = the long run probability of rejecting a true null hypothesis
• alpha = critical value of p-value cutoff for experiments. The Type I error we are willing to tolerate.
• beta = cutoff for probability of accepting a false null hypothesis
• Power = the probability that a test will reject a false null hypothesis (1 - beta). It depends on effect size, sample size, chosen alpha, and population standard deviation
• Multiple testing = performing the same or similar tests multiple times - need to correct alpha value

Statistical power

• Type 1 error - \(\alpha\) - incorrectly rejecting a true null hypothesis
  • This is saying that there is an effect when there isn’t
• Type 2 error - \(\beta\) - incorrectly accepting a false null hypothesis
  • This is saying that there isn’t an effect when there is
• Power is the probability of rejecting a false null hypothesis
• Mostly we shoot for a power of around 80%
• Power can be calculated post hoc or a priori

Power | the things one needs to know

\[ Power \propto \frac{(ES)(\alpha)(\sqrt n)}{\sigma}\]

• Power is proportional to the combination of these parameters

  • ES - effect size; how large is the change of interest?
  • alpha - significance level (usually 0.05)
  • n - sample size
  • sigma - standard deviation among experimental units within the same group.

Power | rough calculation

R Interlude | Complete Exercises 3.2-3.4

Parent offspring regression

Linear Models - a note on history

Linear Models - a note on history

Bivariate normality

Covariance and correlation

A linear model to relate two variables

Parameter Estimation | Ordinary Least Squares (OLS)

• Algorithmic approach to parameter estimation
• One of the oldest and best developed statistical approaches
• Used extensively in linear models (ANOVA and regression)
• By itself only produces a single best estimate (No C.I.’s)
• Many OLS estimators have been duplicated by ML estimators

Many approaches are linear models

• Flexible - applicable to many different study designs
• Provides a common set of tools (lm in R for fixed effects)
• Includes tools to estimate parameters:
  • sizes of effects like the slope
  • difference in means among categories
• Is easier to work with, especially with multiple variables

Many approaches are linear models

• Linear regression
• Single factor ANOVA
• Analysis of covariance (ANCOVA)
• Multiple regression
• Multi-factor ANOVA
• Repeated-measures ANOVA

Plethora of linear models

• General Linear Model (GLM) - two or more continuous variables

• General Linear Mixed Model (GLMM) - a continuous response variable with a mix of continuous and categorical predictor variables

• Generalized Linear Model - a GLM that doesn’t assume normality of the response

• Generalized Additive Model (GAM) - a model that doesn’t assume linearity

linear model parameters

linear models in R

• Need to fit the model and then ‘read’ the output

trial_lm <- lm(Y~X)
summary (trial_lm)

• In general you will ask R to fit linear models and then do additional analyses on the output

Model fitting and hypothesis tests in regression

\[H_0 : \beta_0 = 0\] \[H_0 : \beta_1 = 0\]

full model - \(y_i = \beta_0 + \beta_1*x_i + error_i\)

reduced model - \(y_i = \beta_0 + 0*x_i + error_i\)

1. fits a “reduced” model without slope term (\(H_0\))
2. fits the “full” model with slope term added back
3. compares fit of full and reduced models using an F test

Model fitting and hypothesis tests in regression

Relationship of correlation and regression

\[\beta_{YX}=\rho_{YX}*\sigma_Y/\sigma_X\] \[b_{YX} = r_{YX}*S_Y/S_X\]

Residual Analysis | did we meet our assumptions?

• Independent errors (residuals)
• Equal variance of residuals in all groups
• Normally-distributed residuals
• Robustness to departures from these assumptions is improved when sample size is large and design is balanced

Residual Analysis | did we meet our assumptions?

\[y_i = \beta_0 + \beta_1 * x_I + \epsilon_i\]

Residual Analysis

Residual Analysis

Residual Plots | Spotting assumption violations

Anscombe’s quartet | what would residual plots look like for these?

Anscombe’s quartet | what would residual plots look like for these?

Multiple Linear Regression - Goals

• To develop a better predictive model than is possible from models based on single independent variables.
• To investigate the relative individual effects of each of the multiple independent variables above and beyond the effects of the other variables.
• The individual effects of each of the predictor variables on the response variable can be depicted by single partial regression lines.
• The slope of any single partial regression line (partial regression slope) thereby represents the rate of change or effect of that specific predictor variable (holding all the other predictor variables constant to their respective mean values) on the response variable.

Multiple Linear Regression | Additive and multiplicative models of 2 or more predictors

Additive model \[y_i = \beta_0 + \beta_1x_{i1} + \beta_2x_{i2} + ... + B_jx_{ij} + \epsilon_i\]

Multiplicative model (with two predictors) \[y_i = \beta_0 + \beta_1x_{i1} + \beta_2x_{i2} + B_3x_{i1}x_{i2} + \epsilon_i\]

Multiple linear regression assumptions

• linearity
• normality
• homogeneity of variance
• The absence of multi-collinearity
  • a predictor variable must not be linearly predicted by - or correlated with - any combination of the other predictor variables.

checking for multi-collinearity

library(car)
scatterplotMatrix(~var1+var2+var3, diag=”boxplot”)

R Interlude | Exercise 4.1

ANOVA | Let’s start with an example

• Percent time that male mice experiencing discomfort spent “stretching”.
• Data are from an experiment in which mice experiencing mild discomfort (result of injection of 0.9% acetic acid into the abdomen)
• The injected mice were then kept in:
  • isolation
  • with a companion mouse not injected or
  • with a companion mouse also injected and exhibiting “stretching” behaviors associated with discomfort

From Langford, D. J.,et al. 2006. Science 312: 1967-1970

ANOVA | One or more predictor variables

• One-way ANOVAs just have a single factor
• Multi-factor ANOVAs have more than one factor
• The interaction among factors needs to be considered
• Multi-factor ANOVAs
  • Factorial - two or more factors and their interactions
  • Nested - the levels of one factor are contained within another level
  • The models can be quite complex

Multifactor ANOVA | Assumptions

• Normally distributed groups
  • robust to non-normality if equal variances and sample sizes
• Equal variances across groups
  • okay if largest-to-smallest variance ratio < 3:1
  • problematic if there is a mean-variance relationship among groups
• Observations in a group are independent
  • randomly selected
  • don’t confound group with another factor

R Interlude | Exercise 4.2

What are multivariate statitistics?

• General - more than one variable recorded from a number of experimental sampling units
• Specific - two or more response variables that likely co-vary
• Goals of multivariate statistics
  • Data reduction and simplification (PCA and PCoA)
  • Organization of objects (Cluster Analysis and MDS)
  • Testing the effects of a factor on linear combinations of variables (MANOVA and DFA)

Conceptual overview of multivariate statistics

Multivariate statitistics definitions?

• Some definitions first
• i = 1 to n objects and j = 1 to p variables
• Measure of center of a multivariate distribution = the centroid
• Multivariate statistics uses eigenanalysis of either matrices of covariances of variables (p-by-p), or dissimilarities of objects (n-by-n)
• Matrix and linear algebra are therefore very useful in multivariate statistics

Multivariate statitistics conceptual overview

Eigenanalysis

\[Z_{ik} = c_1y_{i1} + c_2y_{i1} + c_3y_{i2} + c_1y_{i3} + ... + c_py_{ip}\]

• Derives linear combinations of the original variables that best summarize the total variation in the data
• These new linear combinations become new variables themselves
• Each object can now have a score for the new variables

Eigenvalues

• Also called
  • characteristic or latent roots or
  • factors
• Rearranging the variance in the association matrix so that the first few derived variables explain most of the variation that was present between objects in the original variables
• The eigenvalues can also be expressed as proportions or percents of the original variance explained by each new derived variable (also called components or factors)

Eigenvectors

• Lists of the coefficients or weights showing how much each original variable contributes to each new derived variable
• The linear combination s can be solved to provide a score (\(z_{ik}\)) for each object
• There are the same number of derived variables as there are original variables (p)
• The newly derived variables are extracted sequentially so that they are uncorrelated with each other
• The eigenvalues and eigenvectors can be derived using either spectral decomposition of the p-by-p matrix, or singular value decomposition of the original matrix

Correlation vs. dissimiliarity matrices

• Principal Component Analysis (PCA) and Correspondence Analysis (CA) use covariance or correlation of variables.
• Principal Coordinate Analysis (PCoA), Cluster Analysis and Multidimensional Scaling (MDS) use dissimilarity indices.
• The scaling of the derived latent variables can therefore differ between analyses that use covariance of variables as compared to dissimilarity indices.
• Whether the derived variable can be considered metric (e.g. on a rational scale) or non-metric (e.g. on an ordinal scale) depends on the analysis.
• As a consequence the downstream use of the derived variables can change depending upon the analysis.

Correlation vs. dissimiliarity matrices

Dissimiliarity indices for continuous variables

How many PC’s or PCo’s should I concern myself with?

What else can I do with the z-scores of the new PCs and metric PCoA’s?

• They’re nice new variables that you can use in any analysis you’ve learned previously!!
• You can perform single or multiple regression of your PCs on other continuous variables (e.g. an environmental gradient).
• If you have one or more grouping variables you can use ANOVA on each newly derived PC.

R Interlude | Exercises 4.3-4.4

• Getting slightly more advanced and bringing it all together